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ABSTRACT 

Affective development is considered an important 
outcome of compensato'iyy progtams. This study examines interactions 
among program,^ cognitive; and affective variables over an extended 
time period. Affective dimensions considered for measurement include 
attitudes toward school, self-concept, achievement motivation, and' 
internal- external ccntro^b. Selecting areas for measurement is a 
critical ta^k. Alternative measurement procedures are standardized ^ 
instruments, available research instruments, or specially produced 
instruments. Each has inherent problems in a longitudinal study. 
Matters of special concern include linearity of affective change, 
comparability of different instruments) tri;^n-i r.a of i n^:^^ ner i^nc^d 



fexaminer^, 'and positive bia: 



trianing of. inexperienced 
of very young students. (Author/DEP) 
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I. RATIONALE FOR MEASURING AFFECTIVE DEVELOPMENT 

The primary thrust of all compensatory education programs is to improve the 
cognitive development of partOjCipating students, A parallel, but subordinate, 
objective is the improvement of students' school -related non-cognitive develop- 
ment. For example, the California guidelines for Title I. state: / 

The i^aln^goal of every Title I project shall be to increase the ^' 
academic" achievement level of all eligible project participants 
to reflect a normal range and dis tributiorh of academic achievemapt^' 
for the target population- as compared 'with the "general population. 
Enhancing pupi.1 self-image,- motivating the pupil to achieve, - 
improving his healt^ and raising hi^s aspiration levels are. to be 
considered supportive ob3ectives that must be attairfed in order ^4:o 
meet: the project goal. ' * - 



Thus, while cTbgnitive- growth is the basic goal of a Title, I project, non- 
cognitive improvenent is viewed ^as an intervening variable — a precondition 
that enhances the likelihood of achieving cognitive growth, and that is 
influenced in turn by success or failure of achievement in the -cognitive area. 
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School personnel involved in compensatory programs typically express great 
interest"* in the area tha-t i^broadly termed "attitude toward school" and' that 
ir\cludes,a student's feefings abou^t peers, teachers, instruction, and learning. 
Most compensatory educatic^^ pragrair. descriptions include a concern with the 
improvement in students' liking''€or different aspects of school cwork, partlcu^ 
larly likmg for activities in readrng and nathemat lcs , (which are the mai,n. 
areas of concern in .compensatory education), • . - 



^Paper presont^.-d at the AERA Convention', San Francisco, Ca*. , April \9-23^ 197 
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As part <pf the Sustaining Effects Study, SDC will measure students* affdttive 
"development. Th^ longitudinal nature of the study provides an opportunity to 
measure changes in affective behavior that occur as a student experiences the, 
.Qumulative effects of a compensatory education program. 'SIXT will fa^concerned 
with measuring the extent to which a student's affective development is ^ 
inflxienced by participation in a compensatory education program, and the 
differential effects oa affective behavior of programs with iden£ifiably 
different characteristics. While direct assessment of affective development 
would be preferred^ (e.g., through observation of studentrs) , this is precluded 
by the saze of the sample, the burde^h that would be imposed on students, and 
funding 'Constraints. Thus, the measuretnent task must ^be accomplished less j 
directly, through self-report instrumecHt^ completed by studen^t's in the study. 

it 

II. METHODS OF MEASURING AFFECTIVE DEVELOPMENT ' \ 

There are two methods of obtaining instruments for measuring affective-^ behavior. 
One IS to use existing instruments^ and. the second is to develop new instru- 
ments, either by creating new items or by selecting items from other scales. 
Each method has been used in earlier studies of compensatory education 
programs, aod each method is accompanied by serious problems. . 

'In its evaluation of Follow Through, Stanford 'Research Institute has used a 
variety of existing instruments. Generally, the results have pointed to limi- 
tations in the discriminatin^g power of the instrument^ and a lack of strdng 
relationships with program characteristics. 



Both the Equality of Educational Opportunity study and the Emergency School 
Aid Act study employed the technique of emBedding £^ffective items within a ^ 
questionnaire completed by studfents. In both cases, the scales formed by the 
items were not successful in distinguishing between programs with different 
characteristics. t. 
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In a more narrowly focused study, Educational Testing Service evaluated Com- 
pensatory Reading Programs and, having found none of the available instruments 
adequate to the study, developed inventories to measure students' attitudas^ 
•toward reading. Early ^^sults indicate that these inventories are capable^ pf 
distinguishing among different reading programs;' ^ 

The major problem with usihg existing instruments has been that they may not 
be Appropriate to the needs of a' given" study . Typically such instruments were 
developed for particular respondents and specific purposes, and their utility 
for other kinds of respondents is likely tq^ be limited. The major problem 
in de\relopin.g_ne\^ instruments for a particular evaluation effort is the time 
and expense involved in determining the reliability, validity, and other 
psychometric propert;ies of test items. . , - ^ 



III. AREAS OF INTEREST AND CRITERIA FOR SELECTING INSTRUMENTS 

In investigating the. feasibility of using existing instruments or developing 
new ones in the Sustaining Effects Study, ^DC. first identified areas within^ 
the affective domain,, that appeared to be relevant to the evaluation study. 
Investigators Idoked for, areas of concern to an evaluation of compensatory 
education; areas that have been mea-sured xn previous large-sbale evaluation 
Studies; and areas that current ^research has shown to ^e related to students' 
expediences' in school. Application bf these criteria resulted in the selection 
of "attitude toward school," "self-concept /'locus of control," and ^'achieve- 
menj; motivation" as the areas with the greatest relevanG^e to the study. 

Next, eight ^criteria for evaluating existing instrtaments were developed and 
applied to air instrtiments that had potential for use ir\ the study. In ah" 
ipitial screening of. possible instruments, sta^ndard references on affective 
measures and the.^iles at the'Center for the Study 'of Evaluation at UCLA were 
examined, and the eight, criteria were applied to, all instruments. From an ' 
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original set o-f over 60 instruments, 12 were judged to be of sufficient merit 
to warrant further consideration by SDC, with advice from a panel of experts. 

The eight criteria were: 

1. Validity and reliability ^ There should be information available (in a 
manual or in research literature) indicating that the instruirvent has 
Acceptable construct validity and reliability. 

2. Interpretability . Scores generated by the instrument should be easy to 
interpret for their underlying affective dimensions and should not require 
complicated or awkward interpretations. 

3. Age appropriateness . The instrument should be valid for some or all of 
the ages of students in the study. 

4. Administrative ease . Relatively naive examiners (in the case of this 
study, public school teachers) should be able ro administer rhe instrument 

' after limited training. ' ^ 

5. Scoring ease. Because of the large number of students to be measured, an 
.instrument should be designed for, or lend itself to, machine scoring 
procedures. 
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6. Brief testing time . Because the amount of time available for measuring t 
affective behavior i.s limited in the study, an instrument should not 

\ require extensive time to administer. 

7. Minimal ^ rcs^yonse bias . Younger students often demonstrate a bias toward ' 
socially desirable responses, and instruments should be designed to minimize 

''this bias. This can be accomplished by both the manner in which items are 
prepared and the type of response called for. ' y 

4' 

8. Commonality across grades . Because of the longitudinal nature of the study, 
either the same or highly related instruments should be used. In particular, 

^ "instruments prepared^ with parallel versions to en-cjpmpass different grade 
levels were preferred. ' 
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As noted earlier, 12 instruments survived the initial screening. These instru- 
ments are described here briefly. 

AnimalXcrack^rs (Adkins and Ballif) . In its development form, known as 

Gumpgookies. Measures achievement motivation. Developed for preschool 
and primary-grade students. Requires 30-45 minutes testing time. 

M-Scale (Williams). Measures achievement motivation. Developed for upper- 
grade students'. Doubtful validity. Requires about 10 minutes. 

Self-Estey4m Inventory (Coopersmith) . Measures attitude toward self in several 
dom^ns. Developed for upper-grade 'students ; has been used in primary 
grades in national studies. Requires about 10 minutes. 

Self-Concept of Ability (Brookover) . Measures academic self-concept. Devel- 
*oped at secondary-school level, modified for use at primairy level. 
Requires about 20 minutes. ' 

Children's Self-Concept Scale (Piers-Harris) . Measures concerns children have 
about themselves. Developed for upper-grade studentg. Requires 15-20 
minutes . 

Intellectual Achievement Responsibility Questionnaire (Crandall et al.) . 
f^asures control over, and responsibility for, intellectual-academic 
success and failure. ' Developed for upper-grade students; has 'been used 
at primary grades in national studies. Requires 15-20 minutes^. 

Locus of Control Scale for Children (Nowicki and Strickland) . Measures 
generalized expectancies for internal versus external control of 
reinforcement. Developed for middle and upper grades. Requires about 
15 minutes. - '^^^-^ ^ 

Children's Locus of Control Scale (Bialer-Cromwell) . Measures generalized 
40CUS of control. Developed at all elementary-grade levels:. Doubtful 
validity. Requires about 15 minutes. 
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Attitude Toward Learning (Roshal et al.) . Measures general attitude toward 
learning at school. Developed at upper elementary level. Complicated 
response format. Requires about 25 minutes. r 

Attitude Toward School (Roshal et al.). Measures attitude toward school in 
general. Developed at upper elementary l^vel* Complicated response 
format-*. Requires about 25 minutes. 

. Attitude Toward Reading (ETS) . Measures attitude toward reading instruction 
and reading-related activities. Forms for primary level and upper-grade 
level. May induce positive response bias. Requires^ 20-30 minutes. 

Quality of School Life (Epstein and McPartland) . Measures satisfaction with 
school, commitment to classwork/ reactions to teachers. Developed for 
upper-grade students. Requires about 20 minutes for full scale. 

In /.e table that follows, the 12 instruments that survived the original 
screening are shown # along with an indication of how each instrument fared 
with the eight criteria described earlier. An "X" indicates that the 
instrument was judged adequate on that criterion, means that there was 

some doubt about the instrument for that Criterion while a blank indicates 
"serious concern. 
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IV. RECOMMENDATIONS FH6m PANEL OF EXPERTS 

A panel of experts on measuring affective behavior was convened to make recom- 
mendations to SDC. The panel met at SDC on September 10-11, 1975, and con-- 
sisted of: 

Dr. Joyce Epstein, Center for Social Organization of Schools, 
Johns Hopkins University 

Dr. John Kitsuse, Department of Sociology, University of California, 
Santa Cruz 

Dr. Melvin Seeman, Department of Sociology, University of California, 
Los Angeles 

Dr. James Vasquez, Far West Laboratory -for Educational Research 
cind Development 

uaring the two-day meeting, the panel discussed issues related to the measure- 
'"^m^nt of affective behavior in the Sustaining Effects -'Study , exaanined instruments 
designed to assess affective behavior, and developed a set of recommendations 
for the SDC staff. The ma jor , recommendations were: 

1. Us^ available instruments" rather th^in develop new ones. The panel felt 
that while existing instruments alT suffered from some shortcomings, they were 
undoubtedly superior to any that could be developed by SDC in the brief time 
p]^J?r to data collectic^n .\ 

2. Read all instruments ho all students, in recognitic^h- vpf the obvious fact 

that the intent is to assess affective dfmensions without contamination by 

reading ability. While most instruments are intended to be read by the 

respondent if the respondent' raas completed third grade, the panel felt that 

many upper-grade students would lacJc^suf ficient skills to handle the reading 

' \ 

task,s required; this can be overcome by having instruments read to all students 

* * «, 

regardless of grade level. 

^\ ■ r 
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3. Measure affective behavior before measuring cognitive behavior. In con- 
sidering the total testing schedule for students in the Sustaining Effects 
Study, the panel strongly urged that affective instruments be administered 
prior to cognitive instruments (i.e., mathematics and reading achievement 
tests and measures of functional literacy). The panel felt that students' 
attitudes would be strongly influenced by immediate occurrences, and that if 
the affective instruments were administered late in the week of testing, 
students might express negative views that would not be truly characteristic 
of them but rather would be in response to the (potentially frustrating) 
achievement test experiences. 

4. * Measure students* sense of change over time. _ The panel suggested that a 
valuable addition to the assessment of affective tJBhavior would be 'the use of 
itepis-that asked 'the* student to indicate the extent to which the student was 
aware of improvement in his skills and Changes in his feelings and attitudes. 
In. proposing this addition, the panel noted that the longitudinal nature of 
the study included the repeated measurement of students' affective behavior 
and allowed for the actual consideration of change over time. However, the 
paneT suggested that a particul^ly useful piece of information would be the 
student's own awareness of that change. The panel suggested the developroent 
of additional items that would indicate the extent to which a student recog- 
nized that his skills in , reading and mathematics had improved, and that his 
feelings about himself in the school setting had altered with the passage^ 

of time and the experiencing of certain Educational activities. 

V. ' INSTRUMENT TQ BE USED IN THE STUDY 

Following the panel meeting, members of the SDC staff met with personrfel from 
USOE to formalize decisions about measuring affective development. The most 
critical decision, based on the panel's concern with the adequacy of the 
recommended instruments, was that the study should focus primarily on ,the 
measurement of attitudes toward mathematics and reading. Inasmuch as the 
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principal objective of compensatory^ education programs is to improve skills 
in reading and mathematics, it was judged most appropriate for the Sustaining 
EfJ^cts Study to be especially concerned with students* attitudes toward those 
two curricular areas. Given thrs decision, a search was instituted to locate 
the best existing instrument for attitudes toward mathematics and reading. 

The search resulted in the selection -of the "Survey of School Attitudes" 
(Harcourt Brace Jovanovich, 1975) for^ use in *the study. This instrument has 
been recently developed and released, and is ideally suited to the needs of 
the Sustaining Effects Study. The "Survey of School Attitudes" (SSA) is 
designed to\ measure student reactions to reading and other language arts, 
mathematics, science, and social studies. Students indicate whether they lik^, 
dislike, or are neutral toward differenl: activities in each curricular area. 
Thex Survey can be used in group administration settings by a classroom teacher. 
There* are two levels of the Survey: Primary (grades 1-3) and Intermediate 
(grades 4-8) » the two scales of interest in the Sustaining Effects *Study , 

the following substantive topics are included: 



Reading and other language arts: re<flding, working with words anjd 
sounds, writing, speaking, listening. 

Mathematics: concepts '(of numeration, sets, etc.), computation, 
geometry and measurement, problem solving, charts and graphs. 

The SSA was standardized in 1973 on a sample of 13,500 students in grades one j 
through eight. Twelve school systems in 10 states participated in the 
standardization. The standardization sample was highly similar to this 
nation's population, a^ indicated by 1970 census data, on the following 
dimensions: geographic region, socioeconomic variables, minority population, 
and community size-. 
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Item analyses of the mathematics and reading scales yielded the following: 
Median Item-Scale Correlations (All Grades Corokufted) 



Form A, Reading 

Form A, Mathematics 

Form B, Reading 

Form B, Mathematics 



Reading 

• 54 

• 42 

• 56 

• 41 



Mathematics^ 

• 37 

• 57 

, •38 

• 57 



Reading-Mathematics Correlations 



Primary Level 
Inteigiiediate Level 



Form A 

• 69' 

• 43 



Form B 

• 69 

• 44 



These data demonstrate that items correlated considerably higher with their 

/ 

owfi scale than with the other scale, *and, that there is some degree of 
commonality in the measuremgnt of attitudes toward reading and towards 



mathematics . 



RaliaJbilities of the instrument were determined both by Cronbach's Coefficient^. 
Alpha procedure and by test-retest with alternate forms over a 10-day interval. 
Results are summarized below: ^ (# 

\ 
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Test Form 

Primary Reading, Form A 
Primary Reading, Form B 
primary Math, torm A 
Primary Math, Form B 
In;termediate Reading, Form A 
Intermediate Reading, Form B 
Intermediate Math, Form A 
Intermediate Math, Form B 



Alpha Coefficient's 

• 81 
.83 

• 85 

• 85 

• 84 
.82* 

• 90 



" Scale ^ 

\ 

Primary Reading 
Primary Math 
Intermediate Reading 
Intermediate Math , j 



Test-Retest Coefficients 

• 65 
.65 

\ .83 



jl^ese reliability estimates indicate that the SSA yields reasonably stable 
scores • 



Validity of the instrument was approached from several directions. TJo find 
out whether the instrument actually measures student achievement, correlations 
between SSA scores and achievement test scores wera computed. They, seldom 
exceeded .30, leading to the conclusion that the SSA'*' measures, something 
different from achievement, and is npt overly influenced by achievement. To 
find out whether students respond in socially^desirable directions*^ or to 
please the tqacher, scale intercorreiations were inspected to see if they ''^ 
exceeded scale reliabilities (they did not) , and teachers were asked whether 
they felt students responded honestly (they did).. These results point to a- 
lack of response bias. Finally, construct validity was approached through a 
series of factoi? analyses, which showed the SSA to measure^ separate attitude 
dimensions. - . ' ^ % 
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To consider minority-group concerns, a substudy vas conducted in which minority 
and norf-minority students' scores were compared. The two groups produdWi 
comparable ^results for ^both reading and mathematics scales, and on both primary 
and intermediate levels. 
« > 

Each item of the SSA contains a picture of an activity related to a curricular 
area, and^as an accompanying statement describing the picture. At the Primary 
level t|le statements are, to be read by the examiner; at the Intermediat,e level 
the statements are also printed on the item.^ .The response options are three 
faces, one smiling, one frov/nmg, and ^One neutral. By marking *the smiling 
fac$, the student indicates liking'for the activity; marking the frowning ^ 
fape indicates dislike for the activity; and the neutral face means the 
student is n^ot sure how he or she feels about the activity,, 

Somt modifications of the existing instrument were made for this study. Since 
science and social studies are not relevant, those scales were eliminated. 
Scales tot reading and mathematics were extended by combining items from the 
alternate forms of the Survey. The two scales contain 20 items, which should 
be sufficient to asse^s^jchanges th^t take place during the' five y^ars of data 
collection in the S'tudy. 

In addition, new items were added to the instriim'ent to 'create scales that 

measure the following: attitude toward school in general, self-concept in 

f- ' 
the school setting, and students* sense of 'change in th.^ir own affective 

behavior pver time. These additional items are described in detail later in 

this .paper. The new items, 'along with the extension of the reading and 

mathematics subscales , ^make tl;ie length of the total instrument about the 

"s^ame as the original SSA v/ith all four subscales. 
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.The Student Affective Measures for the Sustaining Effects Study have two 
levels: a Primary version^ for grades 1-3 and an Intermediate, version for 
grades 4-9. Each version contadns 56 items ^, ^s follows: 

flumber of Items ' 

,■■/■-. 



R5ADINC?/L^NGUAGE ARTS * 
Beading 

Working With Words 
Writing > 
Speaking 
Listening 

Other Related Activities' 

MATHEMATICS-- 

Concepts of Numeration, Sets, eto, 
i 'Computation ' - * ' 

Geometry and Measurement 
Problem Solving 
Charts and Graji'hs 

SCHOOL IN GENERAL 

SELF-CONCEPT IN SCHOOL 

CHANGE OVER TIME 

Sch6ol in General 
> ^ S^lf-Concept y 
Mathematics 



Reading 



TOtAL 



Primary 

1 

2 J 
3 

6 

. 4 
1- 
0 

4 . 
4 ' 

. 2 
2 
2 
_2 
56 



Intermedicite 
.5 

. , 7 

2 
2 

1 ' 
3 

9 
4 

? 

— ^ 1' 
* . 4 
4 

2 
2 

56 . 



The 20 items for the scale on attitude toward mathematics and the 20 items 
for the' reading attitude scale were obtained by using all 1*5 items fforn 
Form A and five it;emS front Form B. Items chosen from Form B are those 
that bear the least resemblance to items in Form A, to minimize the extent 
to which students will feel that they have already answered an item. , Reading 
and mathematics items alternate-, somewhat reducing the development of response 
set by students. 

To measure attitude toward school in general, fowr items from the satisf actiph- 
with-school scale of the "Quality of School Life*' inventory are us^d. The four 
items reflect attitudes that are not tied to subject-matter areas, adults In . 
school, or other students, but rather deal witb school and class in a general 
manner. The items were modified to allow them -to fit a ^response pattern of 
"Yes,"*"Not Sure," and "No," using tne same three faces that appear in the 
reading and mathematics scales. Also, items cast m the negative were changed 
^ to the positive to eliminate >dif f iculties students have rresponding to negative 
items (does one answer "Yes" or "No" to the statement, "I ,do^ not like school ^ 
very much"?) . The items are: 



I enjoy the work I do in clasa- 
School work is very interesting^; 
I like school very much. 



I am happy when I 'am at school ^^^T 



Self-concept in school is assessed by, four items Selected from existing selfr 
concept scales (Piers-Hams "Children's Self-Concept Scale^' and Sears "Self- 
Concept Inventory") and modified, to use the same response pattern ("Yes,"'' 
"Not'"Sure," "No'") . T^e particular items chosen refer specifically to the 
student in the s^rhool setting^", > rather, than being related to self-concept in ^ 
broader context3 ^such as home or play. The items are: 
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* . I Ccin think up answers to questions. ^ 

I like 'to learn about new things. - . - 

I am good in my school work. 

* « 

i can lea|Ti things quickly. 

The eight item^ m^suring the student's sense of change over time in his 
affective behaviof were prepared following guidelines suggested by the expert 
panel'. Tliey, too, Were written to use the response mode described' above . Th 
items are: 

I like reading more than I used to. 

I do better work in readir.g than I used to. 

I like mathematics more than I used to. 

I do ^-better work in mathematics than I used .to. 

School work is more interesting than it' used to be. 

I like school more - than I used to. 

I am better student than I use^ to be. 

I like my school^St/ork more than I used to. 

A carefyl examination of existing locus -of-eorttrol scales failed, to uncoVer 

a reasonable ^ set of items that conld be included in' the affective measares 

instrument for the Sustaining Effects Study- The better scales (e.g., the 

Crandall et al . "Intellectual Achievement Responsibility Questionnaire") are 

cast in the form of 'an' outcome' for which the respondent is to cboose one of 

two possible causes. For example: "If something is easy to , learn at school, 

it is because (a) you pay attention^ (b) the teacher gives you lots of help." 

Using items of this type woulcj require a change in the instrument's response 

format, adding ta the time needed for administration. In addition, younger ^ 

students may experience difficulty in responding accurately^ to such items. 
> « 

,0n ti^e other hand, locus-of -control i t,ems cast in the "yeg-^no" format (e.g., 
"WheVi.I do good work in class it is because I am lucky") are not the best 
'indicators of internal -external locus ^of control. 
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VI. SPECIAL CONCERNS IN A LONGITUDINAL STUDY 

In the Sustaining Effects Study, SDC will address the lisual matters in the 
assessment of affective development, such as va^^dity, reliability, response 
bias, item discrimination, scale qualities, and administrator capability. But 
in a longitudJLnal study there are a number of unique questions requiring 
attention. Three particular ones have been identified at present. 

The first concerns the linearity of affective development. Do students* 
attitnSeS^^de^v^op as a continuous process essentially equal increments, or 
IS that developrn^t characterized by discontinuities with marked plateaus 
'and peaks? Tne stuo^NS twice-a-year testing schedule of the same students 
for a number of veaTs should provide information relevant to this question, 
but care musT: be exercised m analyzing results to guard against drawing con- 
clusions based on ' fortuitous nappenings that are not, reff lective of the actual 
development of affeccive behavior. 

A second concern is wj^th the u^e of repeated measurements with the same instru- 
ments.^ Whiles the scales for attitudes toward reading and mathematics are 
made up of different, items in the primacy and intermediate versions they are 
extremely sirjilar, arid identical items are used in both versions for scales 
of attitude toward school, self-concept, and sense of change in attitudes. 
It is possible for a -student entering the study, m the fourth grade and tested 
for five years to respond to tne same'^items ten times. What will be brec^by 
such an occurrence: contempt, indifference, boredom, frustration? Will sheer 
familiarity result in stude'nts responding m ways that are not representative 
of their true attitudes? Data on ijit'ra-individual responses ' must be inspected 
to cast .light on this issue, and it is likely that special observations of 
testing sessions will be carried but to collect ^anecdotal information on 
^Student behavior durin[g administrations of the Student Affective Measures. 
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The t±iird issue concern^the interrelationships among program, cognitive, 
and affective variable/. The essential questions are, do program character- 
istics Unfluence Simultaneously both cognitive and affective development, 
does cognitive deve^^oment influence affective development, or does affective 
^ development prodvce changes^m cQgn.itive development? Valid arguments have 
""^b^en offered for all three positions, and a critical .analytic is^ue in this 
study will be to tease out conclusions related to the interactions among 
different ' variables . Should the longitudinal results clearly point to a 
particular statement about the interdependence of program characteristics, 
affective chanca$^ and cognitive growth, this statement would have important 

i 



policy inpl;j;^tA*Qi^"'^^5^^ program specifications, 
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